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METHODS AND APPARATUSES FOR TRANSFERRING DATA 



FIELD OF THE INVENTION 

The present invention relates to the field of multimedia data 
5 transmission. In particailar, the present invention in one exemplary 

embodiment relates to multimedia data transmission of real-time transfer 
protocol (RTF) packets using real time streaming protocol (RTSP) in a computer 
network environment. 



10 INTRODUCTION AND BACKGROUND OF THE INVENTION 

Methods of transmitting data are commonly known and performed 
today on a routine basis to send various multimedia data such as text, graphics, 
audio, video, images etc. across computer networks situated in various parts of 
the world. Generally the transmission process requires both hardware and . 

15 software for performing its function. Typically, the hardware includes various 
types of personal computers and hand held multimedia data sending or 
receiving devices. These devices run under the control of an operating system 
and utilize multimedia application software programs. As is known in the art, 
streaming media data is data which is transmitted to a receiving computer 

20 system and presented (usually after buffering temporarily at the receiving 
system) and then discarded (not stored) at the receiving system. 
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Currently, data is sent in form of packets from one multimedia device to 
another. A large amount of information is required to be sent in a real-time 
manner in the data packets, which imposes a heavy load on the systems. 
Streaming media data, such as Real- Audio data in streaming media format 
5 specified by Real-Networks, is sent through the Internet is near real-time 
manner in many cases. 

In one approach, the components involved in data transmission of 
streaming media are known to be a server (which may be referred to as 
originating server), a caching proxy server and a client. These components in 

10 various combinations commimicate with each other for transmitting data 

packets in real-time. The communication link that currently exists between the 
components uses real-time transfer protocols (RTF) and real-time streaming 
protocols (RTSP) to commimicate and send packets to each other. For this 
approach to work, a caching proxy server needs to commimicate with the 

15 system server, receive a stream of RTF data packets, and transfer the 

information contained within the RTF data packets to a client. Figure la shows 
an example of a prior method in which a caching proxy server receives 
streaming media data and provides this data to a client. In order to perform its 
function properly and efficiently, the caching proxy server needs several pieces 

20 of information from the server to be able to cache an RTF stream easily and 
reliably. 
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A problem with the current approach is that it is not able to provide 
some of the key required information such as data packet transmit time and 
video packet frame type information that a caching proxy needs to be efficient. 
This information allows a caching proxy server to provide smooth packet 
5 delivery to its client by knowing the time an RTP data packet was intended to 
be sent, and type of video frame that is being sent without knowing the specific 
payload format. Another problem with the current approach is that it is not 
able to provide multiple pieces of imrelated data in one delivery to the caching 
proxy server. Furthermore, packets from the server may be 'lost'' and never 

10 reach the caching proxy server. In addition, there is normally no way to 
recreate a complete "pristine'' copy at the caching proxy server. 

Prior art servers communicate RTP information to the caching proxy 
server by sending information through a cache-control header. In one 
approach, a cache-control header contains normal header fields. In another 

15 approach, unrelated to cache control of RTP information, a single type of 
additional information has been added to the normal fields in a header 
extension format without specifying the type of additional information. In this 
approach only a single piece of RTP extension can be added to the normal field 
of the header and sent at any one time. 

20 A problem with using this limited, non-extensible approach is that a 

server is not able to attach multiple sets of unrelated data at a time to send to 
the caching proxy server. Another problem with this approach is that the 
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header extension used in these methods are still not able to provide all the 
information a caching proxy server needs to cache a stream properly and to 
transmit the stream properly. Yet another problem with this approach is that 
there is no way to identify the particular extension independently of other 
5 possible extensions. 
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SUMMARY OF THE INVENTION 

The present invention provides several methods and apparatuses for 
transmitting multimedia data using streaming media protocols such as real- 
time transfer protocols (RTP) and real-time streaming protocols (RTSP) in a 
5 computer network environment. In one exemplary embodiment, a request for 
RTP data is sent from the caching proxy server to the server. The request may 
be for one specific type of data and its related extensions or multiple unrelated 
types of data and their related extensions. The server responds to the request 
indicating its support for the requested RTP data. The caching proxy server 

10 determines whether to proceed or terminate the data transmission process 
based on the response provided by the server. If it is determined to proceed 
with the data transmission process, the caching proxy informs the server to 
send the requested and supported RTP data. The server sends the requested 
data in a variable and extendible header format. 

15 In another embodiment, the caching proxy server requests and receives 

packet transmit time data and / or packet frame type data from the server. The 
caching proxy server uses the frame type data to communicate with the client 
and supply frames based on client's capacity to handle loads at given times. 
Transmit time data is also used by the caching proxy to store packets locally 

20 and deliver these packets at appropriate times to the client for a smooth packet 
delivery. 
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Other features and advaritages of the present invention will be apparent 
from the accompanying drawings, and from the detailed description, which 
follows below. 
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BRIEF DESCRIPTION OF THE DRAWINGS 

The present invention is illustrated by way of example and not limited 
by the figures of the accompanying drawings in which like references indicate 
similar elements and in which: 
5 Figure la is a flowchart which shows a method in the prior art for 

transferring streaming media data to caching proxy server and then to a client. 

Figure lb illustrates a network of computer systems in which media data 
may be exchanged and /or processed, according to one embodiment of the 
present invention. 

10 Figure 2 illustrates a block diagram of an exemplary digital processing 

system, which may be used in accordance with one embodiment of the present 
invention. 

Figure 3 illustrates one embodiment of a commimication method 
between a server and a client using RTSP and RTF protocols. 
15 Figure 4 illustrates another embodiment of a commimication method 

between a server, caching proxy server and a client. 

Figure 5 illustrates one embodiment of a RTSP, RTP negotiation process 
between a caching proxy and a server. 

Figure 6 illustrates one embodiment of a relationship between the server, 
20 caching proxy, and client during a transfer of a Transmit Time (TT) sub- 
extension to the caching proxy server and its use of TT information in 
transmitting streaming data to a client. 
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Figure 7 illustrates one embodiment of process that takes place during 
transfer of a transmit time sub-extension between server and caching proxy 
server. 

Figure 8 illustrates one embodiment of process that takes place during 
5 transfer of a frame type sub-extension between server, and caching proxy 
server. 

Figure 9 is a flow diagram of one embodiment of an operation to provide 
various types of information to a caching proxy in an extensible header format. 

Figure 10 illustrates one embodiment of a relationship between the 
10 server, caching proxy, and client during a transfer of a Frame Type sub- 
extension. 

Figure 11 illustrates a block diagram of a machine readable medium 
which stores executable computer program instruction for execution by an 
exemplary caching proxy server, which may be used in accordance with one 
15 embodiment of the present invention. 

Figure 12 illustrates a block diagram of a machine readable medium 
which stores executable computer program instruction for execution by an 
exemplary originating server (server), which may be used in accordance with 
one embodiment of the present invention. 
20 Figure 13 illustrates a block diagram of a machine readable medium 

which stores executable computer program instruction for execution by an 
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exemplary client, which may be used in accordance with one embodiment of 
the present invention. 
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DETAILED DESCRIPTION 

A method and system for providing multimedia data transmission using 
real-time transfer protocol (RTF) and real time streaming protocol (RTSP) are 
described. For purposes of explanation, numerous specific details are set forth 
5 in order to provide a thorough understanding of the present invention. For 
example, various computer network system architectures and digital 
processing system architectures are provided for illustrative purposes rather 
than to be construed as limitations of the present invention. It will be evident, 
however, to one skilled in the art that the present invention may be practiced 

10 without these specific details. In other instances, well-known structures and 
devices are shown in block diagram form to facilitate explanation. 

Figure lb is a diagram of a network of computer systems in which media 
data may be processed, according to one embodiment of the present invention. 
As shown in Figure lb, a number of client computer system, one or more of 

15 which may represent one implementation of a receiving system, are coupled 
together through an Internet 122. It will be appreciated that the term ''Internet" 
refers to a network of networks. Such networks may use a variety of protocols 
for exchange of information, such as TCP/IP, ATM, SNA, SDI, RTF, RTSP etc. 
The physical connections of the Internet and the protocols and communication 

20 procedures of the Internet are well known to those in the art. Access to the 
Internet 103 is typically provided by Internet service providers (ISPs), such as 
the ISP 124 and the ISP 126, which may also be connected with caching proxy 
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servers 130 and 132. Users on client systems, such as the client computer 
systems 102, 104, 118, and 120, generally obtain access to the Internet through 
Internet service providers, such as ISPs 124 and 126, which may also be 
connected through the internet with caching proxy servers 130 and 132. Access 
5 to the Intemet may facilitate transfer of information (e.g., email, text files, media 
files, etc.) between two or more digital processing systems, such as the client 
computer systems 102, 104, 118, and 120 and/or a streaming media server 
system 128 which may be considered an originating server from which caching 
proxy servers receive streaming media data. For example, one or more of the 

10 client computer systems 102, 104, 118, and 120 and/or the streaming media 
server 128 may provide media data (e.g., video and audio, or video, or audio) 
to another one or more of the client computer systems 102, 104, 118, and 120 
and/ or the streaming media server 128. Such may be provided in response to a 
request. As described herein, such media data may be transferred in the system 

15 100 according tracks. Such tracks, in one embodiment of the invention, may be 
created according to a specific format of the streaming media data and/ or a 
specific data commvinication (e.g., network) protocol(s). 

The streaming media server 128 is t5^pically comprised of at least one 
computer system to operate with one or more data communication protocols, 

20 such as the protocols of the World Wide Web, and as such, is typically coupled 
to the Internet 122. Optionally, the streaming media server 128 may be part of 
an ISP which may provide access to the Internet and /or other network for 
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client computer systems. The client computer systems 102, 104, 118, and 120 
may each, with appropriate web browsing software, access data, such as HTML 
documents (e.g., Web pages), which may be provided by the streaming media 
server 128. Such data may provide media, such as QuickTime movies or 
5 QuickTime streaming media data, which may be presented by the client 
computer systems 102, 104, 118, and 120. 

The ISP 124 provides Internet connectivity to the client computer system 
102 via a modem interface 106, which may be considered as part of the client 
computer system 102. The client computer system may be a conventional 

10 computer system, such as a Macintosh computer, a "network" computer, a 
handheld/portable computer, a Web TV system, or other types of digital 
processing systems (e.g., a cellular telephone having digital processing 
capabilities). Similarly, the ISP 126 provides Internet connectivity for the client 
computer systems 104, 118 and 120, although as depicted in Figure lb, such 

15 connectivity may vary between various client computer systems, such as the 
client computer systems 102, 104, 118, and 120. For example, as shown in 
Figure lb, the client computer system 104 is coupled to the ISP 126 through a 
modem interface 108, while the client computer systems 118 and 120 are part of 
a Local Area Network (LAN). The interfaces 106 and 108, shown as modems 

20 106 and 108, respectively, in Figure lb, may be an analog modem, an ISDN 
modem, a cable modem, a satellite transmission interface (e.g., "Direct PC% a 
wireless interface, or other interface for coupling a digital processing system. 



12 



such as a client computer system, to another digital processing system. The 
client computer systems 118 and 120 are coupled to a LAN bus 112 through 
network interfaces 114 and 116, respectively. The network interfaces 114 and 
116 may be an Ethernet-type, Asynchronous Transfer Mode (ATM), or other 
5 type of network interface. The LAN bus is also coupled to a gateway digital 
processing system 110, which may provide firewall and other Internet-related 
services for a LAN. The gateway digital processing system 110, in turn, is 
coupled to the ISP 126 to provide Internet connectivity to the client computer 
systems 118 and 120. The gateway digital processing system 110 may, for 

10 example, include a conventional server computer system. Similarly, the 

streaming media server 128 may, for example, include a conventional server 
computer system. 

The system 100 may allow one or more of the client computer systems 
102, 104, 118, and 120 and/or the streaming media server 128 to provide media 

15 data (e.g., video and audio, or video, or audio) to another one or more of the 
client computer systems 102, 104, 118, and 120 and/or the streaming media 
server 128. Such data may be provided, for example, in response to a request 
by a receiving system, which may be, for example, one or more of the client 
computer systems 102, 104, 118, and 120. 

20 Figure 2 is a block diagram of an exemplary digital processing system 

which may be used in accordance with one embodiment of the present 
invention. For example, the digital processing system 250 shown in Figure 2 
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may be used as a client computer system, a streaming media server system, a 
conventional server system, etc. Furthermore, the digital processing system 250 
may be used to perform one or more functions of an Internet service provider, 
such as the ISP 124 or 126. The digital processing system 250 may be interfaced 
5 to external systems through a modem or network interface 268. It will be 

appreciated that the modem or network interface 268 may be considered as part 
of the digital processing system 250. The modem or network interface 168 may 
be an analog modem, an ISDN modem, a cable modem, a token ring interface, a 
satellite transmission interface, a wireless interface, or other interface(s) for 
10 providing a data communication link between two or more digital processing 
systems. 

The digital processing system 250 includes a processor 252, which may 
represent one or more processors and may include one or more conventional 
types of such processors, such as a Motorola PowerPC processor, an Intel 

15 Pentium (or x86) processor, etc. A memory 255 is coupled to the processor 252 
by a bus 256. The memory 255 may be a dynamic random access memory 
(DRAM) and/or may include static RAM (SRAM). The processor may also be 
coupled to other types of storage areas/ memories (e.g., cache. Flash memory, 
disk, etc.), which could be considered as part of the memory 255 or separate 

20 from the memory 255. 

The bus 256 further couples the processor 252 to a display controller 258, 
a mass memory 262, the modem or network interface 268, and an input/ output 
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(I/O) controller 264. The mass memory 262 may represent a magnetic, optical, 
magneto-optical, tape, and /or other type of machine-readable medium/ device 
for storing information. For example, the mass memory 262 may represent a 
hard disk, a read-only or writable optical CD, etc. The display controller 258 
5 controls in a conventional marmer a display 260, which may represent a 
cathode ray tube (CRT) display, a liquid crystal display (LCD), a plasma 
display, or other type of display device. The I/O controller 264 controls I/O 
device(s) 266, which may include one or more keyboards, mouse/ trackball or 
other pointing devices, magnetic and/ or optical disk drives, printers, scanners, 

10 digital cameras, microphones, etc. 

It will be appreciated that the digital processing system 250 represents 
only one example of a system, which may have many different configurations 
and architectures, and which may be employed with the present invention. For 
example, Macintosh and Intel systems often have multiple busses, such as a 

15 peripheral bus, a dedicated cache bus, etc. On the other hand, a network 
computer, which may be used as a digital processing device of the present 
invention, may not include, for example, a hard disk or other mass storage 
device, but may receive routines and/ or data from a network cormection, such 
as the modem or interface 268, to be processed by the processor 252. Similarly, 

20 a Web TV system, which is known in the art, may be considered to be a digital 
processing system of the present invention, but such a system may not include 
one or more 1/ O devices, such as those described above with reference to I/O 
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device(s) 266. Additionally, a portable communication and data processing 
system, which may employ a cellular telephone and/or paging capabilities, 
may be considered a digital processing system which may be used with the 
present invention. 

5 In the system 250 shown in Figure 2, the mass memory 262 (and/ or the 

memory 254) may store media (e.g., video, audio, movies, etc.) which may be 
processed according the present invention (e.g. by way of tracks). 
Alternatively, media data may be received by the digital processing system 250, 
for example, via the modem or network interface 268, and stored and/or 

10 presented by the display 260 and/or 1/ O device(s) 266. In one embodiment, 
packetized media data may be transmitted across a data commimication 
network, such as a LAN and /or the Internet, in accordance with tracks. On the 
other hand, the processor 252 may execute one or more routines to use a file 
with one or more tracks, or alternatively, to create one or more tracks, to 

15 process media (e.g., a pre-packaged movie, audio file, video file, etc.) for 
presentation or packetization according to the tracks. Such routines may be 
stored in the mass memory 262, the memory 264, and/ or another machine- 
readable medium accessible by the digital processing system 250. In one 
embodiment, the digital processing system 250 may process media data having 

20 tracks embedded therein. Similarly, such embedded media data may be stored 
in the mass memory 262, the memory 264, and/or another machine-readable 
medium accessible by the digital processing system 250. 
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Figure 3 shows an example of components involved in data transmission 
scenario. An originating server 301 and a client 302 are shown as components 
involved in carrying out transmission of streaming media data using RTF and 
RTSP protocols as one embodiment of the present invention. The originating 
5 server 301 and the client 302 may commimicate directly with each other or may 
communicate through an intermediary such as a caching proxy server. In one 
embodiment, the server 301 and the client 302 may be on separate local area 
networks (LAN). In another embodiment the server 301 and the client 302 may 
be connected through a wide area network. There may be either one or several 

10 clients 302 that are in communication with the server 301 directly or indirectly 
through an intermediary, such as the Internet. The server 301 and client 302 
may interact with each other for sending various types of streaming media data 
in various formats. In one embodiment, the streaming media data may be sent 
in a downstream direction from server 301 to client 302. In another embodiment 

15 the client 302 may send requests and other streaming media data information to 
server 301. 

Figure 4 shows an example of one embodiment of a communication 
relationship between a client 302, a caching proxy server (CP) 401 and the 
originating server 301. There may be several types of connections between 
20 these components, but preferably the client 302 may be in commimication with 
the caching proxy server 401 through an Internet connection, and the caching 
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proxy server 401 may be in communication with the originating server 301 
through an Internet connection 

A caching proxy server 401 may be connected through the Internet with 
a single client 302 or several clients 302. The caching proxy server 401 and its 
5 connected clients 302 may be on the same local area network or may be 

connected through a wide area network. In one embodiment it is preferable that 
the caching proxy server 401 and client 302 or clients 302 are cormected through 
a local area network and in close proximity to each other. An exemplary 
embodiment of close proximity connection may be connection in the same 

10 company etc. where the connection may utilize a high bandwidth interface. The 
commimicational link between the caching proxy server 401 and client 302 may 
be of a variety of types such as direct cable, fiber optic, radio frequency etc. 
These links may change and vary based on the need of a particular client 302 
and advancements in technology. 

15 A originating server 301 and a caching proxy server 401 may 

communicate using a communicational link such as direct cable, fiber optic, 
radio frequency etc. These links may change and vary based on a particular 
need and advancements in technology. The cashing proxy 401 may act as an 
intermediary between the originating server 301 and client 302 to transfer 

20 streaming media data and assist in smooth delivery of RTF packets from server 
301 to client 302. In so doing, a caching proxy server 401 may perform several of 
its own functions. In one embodiment the caching proxy 401 functions may be 
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thinning frames, storing streaming media data locally, and transmitting 
streaming media data at offset times to client 302. In another embodiment the 
caching proxy server's 401 functions may be negotiating with originating sever 
301 for various RTP extension associated with various types of streaming media 
5 data, and receiving or responding to various client 302 requests etc. In one 
embodiment, one of the objectives of a caching proxy server 401 is to deliver a 
pristine and good quality copy of streaming media data to the client 302 and do 
so in an efficient and speedy manner. 

Typically a client 302 may sent a request directly to the caching proxy 

10 server 401. The caching proxy server 401 may then react to the client 302 

request and either fetch the requested items from the system server or responds 
on its own. Its own response may be from a copy of streaming media data 
which has already been obtained from an originating server and which has 
been stored on a storage device controlled by the caching proxy server (e.g. a 

15 local hard disk of the caching proxy server). However the system may also be 
configured for the client 302 to send requests directly to the system server 301 
and have the server 301 respond back directly to the client 302 or indirectly to 
the client 302 through a caching proxy server 401. 

Figure 5 shows one exemplary method according to an embodiment of 

20 the present invention. In the operations of Figure 5, an originating server(e.g. 
server 301) and a caching proxy server 401 communicate with each other to 
assist in smooth transmission of streaming media data. This communication 
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aids smooth packet delivery in many ways including allowing the caching 
proxy server 401 to deliver to the client 302 good quality streaming media data 
at a high speed. In addition, the communication also aids in assisting and 
managing client's load by ensuring that the client 302 gets a manageable 
5 amount of streaming media data and no frames are dropped in the process or 
only less important frames dropped in the process (through frame thirming). 

Initially in operation 501, the caching proxy server requests streaming 
media data from an originating server. The request may be made by asking the 
server 301 for "setup'' in RTSP for audio or video streaming media data. The 

10 request may be for one type of streaming media data or several types of 
streaming media. The request may be for similar or unrelated types of 
streaming media data. The server 301 receives the request from the caching 
proxy server 401, and the server 301 responds in the manner described with 
respect to operation 502 of Figure 5, The ''SETUP" request in RTSP in operation 

15 501 may be initiated by the caching proxy server 401, independently of a client 
system 302 requesting streaming media data or the request in operation 501 
may be initiated by a client system 302 requesting the streaming media data 
from the caching proxy server 401 which in turn requests the requested 
streaming media data from the server 301 (if the caching proxy server 401 does 

20 not already have the requested streaming media data stored imder its control, 
such as a local hard disk of the caching proxy server 401). The caching proxy 
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server 401 may also log client's IP address for subsequent communication in the 
case where a client initiated the request. 

The caching proxy server 401 and originating server 301 may establish a 
communication process in which the caching proxy server 401 and the 
5 originating server 301 may engage in a negotiation process 502 for 

communicating back and forth in order to aid a smooth streaming media data 
packet transmission. As shown in operation 501, the caching proxy server 401 
may commimicate with the originating server 301 and request (e.g. by 
specifying names of RTP extensions) a set of RTF extensions associated with the 

10 streaming media data to be sent to the caching proxy server 401. The set of 
extensions requested to the server 301 may be the same as the set of requests 
sent to the caching proxy server 401 from the client 302 (in those cases where 
the client specifies RTP extensions, such as security extensions, for its use). 

The server 301 receives the request for RTP extensions from the caching 

15 proxy server 401. The server 301 may then run its internal processes to 

determine whether the server 301 supports the requested RTP extensions. The 
outcome of this determination may be that the server 301 supports some but 
not all the requested RTP extensions, or that the server 301 supports none of the 
requested RTP extensions, or that the server 301 supports all of the requested 

20 RTP extensions. The server 301 may respond in operation 502 to the caching 
proxy server 401 by informing the caching proxy server 401 of the server's 301 
supported RTP extensions. The server 301 may choose to respond 502 by 
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indicating only the supported RTF extensions or may respond by indicating 
both the supported and unsupported RTF extensions, or the server 301 may not 
respond at all indicating no support for requested extensions. In one 
embodiment the response may be in an echo form or any several other forms. In 
5 one echo form of the invention, the server transmits the names of the requested 
RTF extensions and an associated code for each named extension. 

The caching proxy server 401 receives a response from the server 301 
indicating the supported RTF extensions or both the supported and 
imsupported RTF extensions. The caching proxy server 401 may check to see if 

10 a response has been sent for all the RTF extensions it had earlier requested. 

Caching proxy server 401 may have received none, one, some, or all responses 
to the requested RTF extensions. Caching proxy server 401 may evaluate 
further to check if any of the server 301 unsupported RTF extensions are 
required for streaming media data transmitting process. Required RTF 

15 extensions may be defined as RTF extensions that are necessary for carrying on 
a particular data transmission operation such as frame thinning etc at the 
caching proxy server 401. As shown in Figure 5, operations 501 and 502 relate 
to setup and negotiation for an audio track while operations 503 and 504 relate 
to similar setup and negotiation for a video / image track. 

20 In one embodiment, the caching proxy server 401 may request multiple 

sets of RTF extensions at a time from the server 301, If the RTF extensions 
requested are required and unsupported by the server 301, then caching proxy 
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server 401 may decide to terminate the negotiation process. It may also be the 
case that some of the extensions are supported and some are not. In such a 
situation, if the unsupported extensions are not required for the data 
transmission process then caching proxy server 401 may decide to proceed 
5 further and receive the supported extensiorts and the associated streaming 
media data. In another embodiment the caching proxy 402 may not receive a 
response for any of the RTP extensions requested. In such a case the caching 
proxy 402 may choose to terminate the negotiation process with the server 301. 
If the caching proxy server 402 decides not to terminate the negotiation 

10 process and to request the supported RTP extensions and streaming media 
data, it may send a request to the server 301 to send the streaming media data 
and the associated supported RTP extensions in operation 504. In the example 
of Figure 5, this request for the streaming media data and the associated RTP 
extensions occurs when the caching proxy server 401 sends a "PLAY" 

15 command in the RTSP protocol. 

The server 301in operation 505,responds to the "PLAY" command by 
sending the streaming media data and by sending the requested and supported 
RTP extensions, which is associated with the streaming media data, to the 
caching proxy server 401 in a extended header format. This header may contain 

20 one, two or three similar or unrelated RTP extensions. 

Upon receiving the streaming media data and receiving RTP 
extensions from the server 301, the caching proxy 401 may store the streaming 
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media data and the RTP extensions in a storing facility 601 (e.g. a storage device 
controlled by the caching proxy sever 401, such as a local hard disk of the 
server 401) and terminate the transmission process with the server 301. The 
caching proxy 401 may again reinitiate the negotiation process and repeat all 
5 the back and forth if another request for streaming media is submitted by the 
client 302. This request may be similar or completely different from prior 
requests. Some of the extensions that may be requested by the cashing proxy 
sever 401 may be a transmit time sub-extension denoted by symbol "iriV, or 
frame type sub-extension denoted by symbol "ftry", or packet position sub- 
10 extension denoted by symbol ''papo". Other extensions may also be requested 
(e.g. an extension which is used by the client 302 or server 401 to maintain a 
secure or encrypted or authenticated communication between client 302 and 
server 401). 

For example, in one cycle of its operation a caching proxy server 401 may 
15 ask for three separate RTP sub-extensions one of which may be frame t5^e sub- 
extension denoted by symbol "frty" (used in frame thinning by caching proxy 
server 401 as described below), the other may be transmit type sub-extension 
denoted by "trti'' (used by the caching proxy server 401 as described below), 
and the last may be packet position sub-extension denoted by "papo'' (which 
20 may be used to retrieve lost or missing packets). Let us also assume for the 
illustration of this example that "frty" sub-extension is required for the 
streaming media data transmission process. 'Trty" may be denoted as a 
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required sub-extension due to several reasons. One of the reasons may be that 
the client 302 cannot receive or process the data at a high data rate (and so 
frame thirming is required) and "frty'' sub-extension will assist the data 
transmission process between a caching proxy server and the client 302 by 
5 allowing the caching proxy sever to perform frame thirming and therefore may 
be "necessary". 

The caching proxy 402 may receive the request and communicate with 
the server 301 by sending a single request to the server 301 asking for both sub- 
extensions. Let us assiame further for the illustration of this example that the 

10 server 301 can only support one of the two RTP extensions. The server 301 may 
then send a response back to the caching proxy server indicating which sub- 
extension is supported. 

If the supported sub-extension happens to be only "trti'', or "papo" or 
both but not "frty" then the caching proxy 402 will terminate the negotiation 

15 process between the caching proxy 402 and the server 301. This is because 

''frty" was a required extension to the data transmission process and since it is 
not supported by the server 301, the caching proxy 402 may not proceed 
further. If however, the supported sub-extension happens to be only "frty'', or 
frty and papo, or frty and trti, or frty, papo and trti, then the caching proxy 

20 server 401 may proceed further with the transmission process. The caching 

proxy server 401 in this instance may choose not to terminate the process since 
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the required sub-extension frty is present in the response as supported by the 
server 301. 

Figure 6 shows an example of a method for transmitting packet transmit 
time data which may be used with various embodiments of the present 
5 invention. The server 301 is connected with the caching proxy server 401 by 
way of a standard communication carrying devices such as fiber optic wire link, 
radio frequency communication, cable wire etc. A person having ordinary skill 
in the art will appreciate that any one-commimication device is not essential for 
the data transfer operation in accordance with this invention and that these 
10 commimications devices are interchangeable. It must be clear that it is 

important for the communication devices to allow communication in both 
directions i.e. from server 301 to caching proxy 402 or from caching proxy 402 
to server 301. 

The communication between a caching proxy server 401 and the 
15 originating server 301 may be a direct commimication relationship or there may 
also be other devices such as routers in the Internet acting as intermediaries to 
assist in streaming media data transfer. Tj^ically, a caching proxy server 401 
is located in closer proximity to the client 302 than the originating server 301. 
This close proximity may be within a company, or on a designed local area 
20 network (LAN), or in the same geographic region, whereas typically caching 
proxy server and original system server 301 are further apart. 
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The caching proxy server 401 may have a storage facility 601 to store 
streaming media data 603 and / or the associated RTP extensions 602. The 
storage facility 601 may be a local to the caching proxy server 401 or on an 
offsite from the caching proxy server 401 but in either case the storage is 
5 controlled by the caching proxy server 401. The caching proxy server 401 may 
have a link established to store data received from the server 301 for a periods 
of time in the storage facility 601, and then be able to retrieve the stored data at 
a later time for sending to client 302. In the example of Figure 6, the streaming 
media data 603 and its associated RTP extension (transmit time in this case) are 

10 stored together on a storage device 601. Groups of streaming media data (e.g. a 
packet or a set of packets) are associated with a corresponding designation of a 
transmit time so that each group has a transmit time which specifies when to 
transmit the particular group. It will be appreciated that the streaming media 
data and the associated RTP extension may be stored separately (but still be 

15 associated - e.g. packet No. xxx to be transmitted at time ABC, packet No. xxy 
is to be transmitted at time ABD, etc.. ) 

In the example of Figure 6, the streaming media data is received by the 
server 401 and the caching proxy server 401 receives the transmit time data 
from server 301 and stores it in the storing facility 601. Transmit time data may 

20 be associated with each track of streaming media data. For example, in one 
instance the transmit time at 0 sec 602 may be associated with corresponding 
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streaming media data 603. In operation, in this exemplary embodiment, the 
streaming media data 0 will be sent to a client at transmit time 0. 

Figure 7 shows one exemplary method for using transmit time as an RTP 
extension according to an embodiment of the present invention. In operation 
5 the method suggested in Figure 7 may utilize the system architecture as 

suggested in one of the embodiments of the present invention shown in Figure 
6. 

In one example of the method of Figure 7, a caching proxy server 401 
receives a request from client 302 for streaming media data and then requests 

10 an RTF extension which specifies transmit time information and requests the 
server 301 to send transmit time sub-extension RTP data 701 and associated 
streaming media data. Operation 701 shows the caching proxy server's request 
for streaming media data and transmit time which results from this request. 
The server receives the request in operation 702 as shown in Figure 7. It may 

15 also be the case that a caching proxy server 401 already had received the 

requested streaming media data and its associated transmit time information 
from the server 301 and has stored the streaming media data and associated 
RTP extensions at a storing facility 601. If such, then the caching proxy server 
401 may start responding to clients 302 request without communicating with 

20 the originating server 301 thereby shipping to operations 707 and 708 of Figure 
7. 
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Assuming for illustration of this example that the original server 301 
supports the transmit time information, server 301 will respond back to caching 
proxy server indicating its support of the requested sub-extension in operation 
703. If however the transmit time sub-extension is not supported by the original 
5 server 301, the originating server 301 may or may not respond back to the 

caching proxy server 401 indicating its support for the requested sub-extension 
as shown in operation 709. In the event of an imsupported sub-extension, the 
caching proxy 402 may terminate the negotiation process as shown in operation 
710 with the server 301 and would typically inform the client 302 of the inability 

10 to provide streaming media data. In so doing, the caching proxy server 401 
may first evaluate whether the missing transmit time information is required 
for rimning its processes. If the result of the determination is that transmit time 
information in this particular example is a required element, then the caching 
proxy server may decide whether to proceed or terminate the transmission 

15 process. 

The server 301 in operation 704 sends the transmit time RTP data in an 
extended header format according to the RTP protocol to the caching proxy 
server. The header may consist of the normal header fields, the sub-extension 
character name and a sub-extension ID 704. The sub-extension character name 
20 for a transmit time data may be a 4-character code denoted by "trti". This code 
may uniquely identify and describe the content of the sub-extension as being 
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transit time data. The sub-extension ID may identify the sub-extension in the 
RTF packet. 

A transmit time sub-extension may consist of a single 64-bit imsigned 
integer representing the recommended transmission time of the RTF packet in 
5 milliseconds as shown in operation 704. The transmit time may be offset from 
one another from the start of a media presentation. For example in one sub- 
cycle of operation, a session description protocol document for a uniform 
resource locator (URL) may include a range of 0-729.45 seconds. The client 302 
may make a PLAY request 706 for the video, audio, text, graphics, and images 

10 etc. type data. 

The caching proxy server 401 may receive the RTF data packet 
associated with streaming media data with the transmit time sub-extension as 
shown in more detail in figure 6. The caching proxy server 401 may then store 
the RTF transmit time data locally as shown in Figure 6. The caching proxy 

15 server 401 may then strip off the header ID in operation 705 and send streaming 
media data associated with each track, in operation 707, of transmit time 
individually at offset times to the client 302 allowing the client 302 to carry on 
PLAY operation 708. An advantage of knowing and storing transit time at 
offsets locally at the caching proxy server, it may now be possible for the 

20 caching proxy server 401 to selectively re-transmit data at different intervals to 
the client 302 or respond to clients request to send data corresponding to any 
particular time slot. 
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Figure 8 shows one exemplary method for a stream thirming process by 
a caching proxy server according to an embodiment of the present invention. In 
operation a client 302 and caching proxy server 301 communicate with each 
other to assist in sending and receiving streaming media data and assisting in 
5 traffic flow control to the client 302. In a method according to Figure 8, a client 
302 commimicates with the caching proxy server 401 and indicates that it is 
overloaded or the caching proxy server 401 detects that the client is overloaded. 
As part of this communication, the caching proxy server 401 ensures that the 
client 302 does not get an amount of data that exceeds its data handling 

10 capacity. Caching proxy server also prevents at least selected frame being 
"dropped'' or missing as a result of an overloaded client 302. 

A principle behind Figure 8 is that an overloaded client 302 may notify 
the caching proxy server that it has reached its capacity for receiving RTF data 
(e.g. streaming media data). The client 302 may have been overloaded due to 

15 several reasons including that a caching proxy server is sending RTF data very 
quickly and the client 302 is having difficulty receiving data at such a fast pace. 
The client 302 may inform the caching proxy server to stop sending streaming 
media data altogether, or to send data at a slower pace. The client 302 may also 
inform the caching proxy server to send only selected order of frames and not 

20 send any low order frames. The caching proxy server 401 will use the frame 
type data to determine which frames to transit to client 302; typically, higher 
priority frames are transmitted while lower priority frames are not transmitted. 
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A method of Figure 8 begins in operation 801 in which a caching proxy 
server 402 may communicate with the originating server 301 and request the 
server 301 for streaming media data and its associated frame type information. 
The frame type identifies various types of data (e.g. frames) in streaming media 
5 data which allows ''thinning" which may be defined as reducing frames, 

sending frames at a slower pace, or not sending certain frames at all. It will be 
appreciated that thirming apples to various types of data and that ''frames'' 
may be considered to be such various types of data. The server 301 may receive 
the request in operation 802 and may respond in operation 803 to the caching 

10 proxy server 401 indicating whether the server 301 supports the requested 
frame type streaming media data. If the server 301 supports this, the server's 
301 response in operation 803 includes sending the associated RTF frame type 
sub-extension in a format described in block 804 along with an identifier code 
corresponding to the frame type extension requested by name in operation 801. 

15 If the server 301 does not support frame type sub-extension then the 

caching proxy server may terminate in operation 807 and 808 the 
commimication with server 301. The server 301 may indicate that it does not 
support the requested frame type streaming media data by either responding or 
not sending any response to the Caching Froxy server 401 which would also 

20 indicate no support of the requested RTF extension for the streaming media 
data. However, if the server 301 supports the frame type sub-extension, the 
caching proxy 402 may inform the server 301 to send the streaming media 
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associated with the frame type information. In one embodiment, the server 301 
may send the supported streaming media data sub-extensions without any 
further requests from the caching proxy server 401. In another embodiment, the 
server 301 may wait for further a further request from the caching proxy server 
5 401 to send the supported streaming media data sub-extensions. 

The server 301 may then send the RTF sub-extension in an extended 
header format. The frame type sub-extension may consist of a single 16-bit 
unsigned integer value with several well-known values representing different 
frame types. The well-known values may be ''1" for a key frame, "2'' for a p- 

10 frame, or "3'' for a b-frame where key frame maybe of the highest order and 
most importance, b-frame of the lowest order and least importance, and b- 
frame somewhere between key frame and b-frame in terms of importance. 
There may also be other frames that may be added to this format. 

The caching proxy server 401 may then store the streaming media data 

15 and its associated frame type sub-extension in its storing device 601 after 

receiving them from the originating server 301. This is shown in operation 805 
of Figure 8. The caching proxy server 401 may then enter into a negotiating 
process with the client 302 in evaluating the client's capability at the time to 
handle streaming media data traffic 809. Based upon the result of the 

20 negotiation process 809, the caching proxy server 401 may thin frames (sending 
only selected, predetermined frames) and send streaming media data 
associated with selected frames 806 to the client 302. 
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For example, in one cycle of operation a client 302 may inform the 
caching proxy server that it is overloaded. The client 302 may inform the 
caching proxy server 401 to stop sending frames altogether or to lower the bit 
rate if the transmission falls behind. In the case of lowering the bit rate and 
5 slowing down, the caching proxy server 401 may stop sending the lowest order 
frames of the streaming media data, the b-frame to the client 302. The caching 
proxy server 401 and the client 302 may commimicate further to evaluate if the 
client 302 is still overloaded. In one embodiment, if the client 302 is capable of 
handling the load after thinning of the b-frame then the caching proxy server 

10 may send the client 302 key-frames and p-frames. However if the client 302 is 
still overloaded then the caching proxy server 401 may further reduce the data 
traffic to the client 302 and stop sending p-frames. The caching proxy server 401 
may further evaluate client's 302 data handling capability and determine if any 
more frame thinning is necessary to reduce load on client 302. In another 

15 embodiment the client 302 may directly specify to the caching proxy server 401, 
which frames to send and which frames not to send until a subsequent request 
is sent to the caching proxy server 401 to change sending considerations. 

After a client 302 retains its capability to cache frames, the caching proxy 
server 401 may again start sending the lower order frames to the client 302. It 

20 may again send all the frames at a high speed or send the frames according to 
requests received by the client 302. In the event that the client 302 gets 
overloaded again, the caching proxy server 401 may repeat the thinning process 
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until the client 302 is able to handle caching data again. Figure 10 shows an 
example of how a caching proxy server 401 receives streaming media data and 
its associated frame type (FT) RTF extension data from an originating server 
301 and stores the streaming media data and associated frame type extension 
5 data on a storage device (e.g. a local hard disk of the caching proxy server 401) 
and then uses the frame type data to selectively thin frames of the streaming 
media data which is being transmitted to a client 302. 

Communication between a caching proxy server 401 and originating 
server 301 or caching proxy server 401 and client 302 is carried on using real- 

10 time transfer protocol (RTF) and real-time streaming protocol (RTSF) for 
sending / receiving streaming media data. An originating server 301 sends 
streaming media data packets in a streaming media format using RTF to a 
caching proxy server 401 whenever a transmission of streaming media data 
occurs. One of the embodiments of the present invention is to be able to modify 

15 the current existing RTF headers by being able to expand the header with sub- 
extensions and also be able to make the header format variable. Expansion of 
the header is useful because a caching proxy server 401 may need several pieces 
of information along with a RTF packet that will aid in providing a good 
quality streaming media data packet and smooth delivery to the client 302. The 

20 extra information that may be needed can be provided by attaching it to the 
existing header by being able to expand the header field. It should also be clear 
that variability of the extended header is important because the extra pieces of 
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information needed by the caching proxy 402 may vary each time. To 
accommodate for this variation, the extended header may have the capability to 
change and provide various types of information as needed by the caching 
proxy server 401. 

5 In accordance with one embodiment of the invention, in operation, an 

extended header consists of a normal header fields. A person having ordinary 
skill in the art is aware of the various header fields that are normally used in 
operation. The normal header fields are immediately followed by header 
extension fields. The extension field consists of several sub-extensions. There 

10 may be several header sub-extensions that are imrelated to each other and 

may vary per request of the caching proxy server 401. The sub-extensions may 
have an extension type of "se''. The RTF extension length may be the total 
length of all the sub-extensions and may be defined in 32-bit words thereby 
being in full compliance with the RTF protocol. 

15 The "se'' sub-extension format may be such that a sub-extension ID 

immediately follows the normal RTF header field. The ID may identify the sub- 
extension within the RTF packet. This ID may be a one octet ID generated by 
the server 301 for each individual named RTF sub-extension. Each sub- 
extension may also have its unique name that is defined by a four-character 

20 name code. This name code uniquely identifies and describes the type of data in 
each sub-extension. For example, the four character name code for a transmit 
time sub-extension may be "trti'', frame type sub-extension may be "frty" and 
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packet position sub-extension maybe '"papo". This name code is associated 
with the one octet ID (generated by the server 301) so that the caching proxy 
server 401 can identify, form the octet ID the appropriate RTF extension data 
when it receives streaming media data. 
5 In one embodiment of the present invention, the imique name may be 

"frty" associated with streaming media data for frame type information. The 
unique name "frty" may also have an unsigned integer associated with each 
different type of frame. In one embodiment the unsigned integer may be "1" for 
a key-frame, ''T for a p-frame, and ''3'' for a b-frame. A user may also add any 
10 additional frames in the future as need and technology advances and may use 
this header format without any need for much modifications. 

In another embodiment of the present invention, the imique name may 
be "trti'' associated with streaming media data for transmit time type 
information. 

15 In another embodiment of the present invention, the unique name may 

be "papo'' associated with streaming media data for packet position type 
information. 

Figure 9 shows an exemplary method of several aspects of the present 
invention. In a portion 901, a caching proxy server 401 requests streaming 
20 media data from an originating server 301 and also requests by name one or 
more RTF extensions. This request is made using the RTSF protocol. In 
operation 903, the server typically responds back (e.g. of a response would be 
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an echo) a response to the caching proxy server 401 indicating its support for 
the requested RTP extensions. The server 301 also transmits to the caching 
proxy server 401 an identifier, such as a number code which corresponds to 
each name of the requested RTP extensions. Typically, the caching proxy server 
5 401 will use the number code later in identifying received extended RTP data. 
The number code allows the caching proxy server 401 to identify the various 
types of RTP extension data in the streaming media which it receives as the 
server 301may not use the name to designate the RTP extension type. In 
operation 905, the caching proxy server 401 receives the server's 301 response 

10 and then in operation 907, the CP server 401 determines whether the server 
SOlresponded to all of the requested RTP extensions. 

If the server 301 did not respond to all requested RTP extensions, then 
processing proceeds to operation 909, followed by operation 911 in which it is 
determined whether any of the missing RTP extensions are critical to the 

15 caching proxy server's 401 processing. If they are not critical, then processing 
proceeds to operation 921. If they are critical, then the caching proxy server 401 
determines in operation 913 whether or not to terminate the 
operation/ comm\mication with the originating server 301. As shown in 
operations 915 or 917, the caching proxy server 401 may terminate 

20 operations/communications with the server 301 for this particular streaming 
media data which was requested or they proceed to receive the streaming 
media and whatever supported extensions can be provided. 
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In operation 921, the CP server 401 requests the originating server 301 to 
send the requested streaming media data and its associated RTP extensions. In 
one embodiment, the CP server 401 transmits a "TLAY" request using RTSP, 
and this causes the server 301 to respond in operation 923 by transmitting the 
5 streaming media data and the associated RTP extensions. In operation 925, the 
CP server 401 stores the streaming media data received from the server 301 and 
also stores the associated RTP extension data. In operation 927, the CP server 
401 may remove certain RTP extension data from the streaming media file, such 
as the transmit time or the frame type data. This is done in order to avoid 

10 sending the transmit time or the frame type information to the client 302 which 
requests streaming media data. The RTP extension data, which is removed 
from the streaming media data, is stored separately but associated with the 
streaming media data. For example, transmit times for various packets are 
stored separately from the packets, but the association existing in the data 

15 received from the server 301 between the transmit time and the corresponding 
packets is maintained even when the transmit times are stored separately so 
that the caching proxy server 401 may determine the appropriate transmit time 
for each of the packets in the streaming media data. In operation 929, the 
caching proxy server 401 evaluates a client's 302 request for streaming media 

20 data and responds accordingly. It will be appreciated that a client 302 will 

negotiate for streaming media data using the RTSP protocol and the CP server 
401 will respond with the streaming media data by transmitting the data to the 
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client 302. In addition, the client 302 may request frame thinning. Further, the 
caching proxy server 401 may use the transmit times to determine when to 
transmit to various packets in the streaming media data to the client 302. 

Figure 11 shows one type of exemplary machine readable media (e.g. 
5 RAM or hard disk or combination thereof ) for storing executable computer 
program instructions for a caching proxy server 401 that may be used in 
accordance with the present invention. The caching proxy server 401 typically 
will have its own operating system (OS) software 1101. This software 1101 may 
be the Macintosh OS. Or Windows NT or Unix, or other well known operating 
10 systems 

The control software 1102 is for transmitting or receiving streaming 
media data using, for example RTF and RTSP protocols. The software 1102 is 
normally able to retrieve or send various types of streaming media data packets 
and direct commands for storing the received media in a storing facility 601. 

15 Thus software 1102 performs the negotiation process with an originating server 
301 and receives streanung media data, and its associated RTF extensions and 
causes the streaming media data and its associated RTF extensions to be stored 
on a storage device controlled by caching proxy server 401. Figure 11 shows the 
storage of two streaming media data files 1103 and 1104. 

20 Streaming media data file 1103 may contain streaming media data 1 in 

streaming media format 1105, transmit time associated with streaming media 1 
(1106), and frame type associated with streaming media 1 (1107). In one 
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embodiment, the operating system 1101 and control software 1102 may have 
the capability to separate streaming media data in packet 1 from other packets 
and store it separately in a storing facility 601 and to extract the RTP extensions 
(e.g. Transmit Time data or Frame T5rpe data) from the stored streaming 
5 media packets and store these separately so that these packets do not include 
the RTP extensions. 

Streaming media data file of 1104 may contain streaming media data 2 in 
streaming media format 1108, transmit time associated with streaming media 
data 2 (1109), and frame type associated with streaming media 2 (1110). 

10 The streaming media data 1105 and 1108 will usually not be in the same 

original format as the media data was at the originating server 301. The 
streaming media data 1105 and 1108 may however be a full ''pristine'' copy of 
the original media data, because the "papo" extension may be used by the 
caching proxy server 401 to search for any missing packets in the streaming 

15 media data 1105 and 1108 and to request (again) these packets from the 
originating server. 

Figure 12 shows one type of exemplary machine readable media (e.g. 
RAM or hard disk or combination thereof) for storing executable computer 
program instructions for an originating server 301 that may be used in 

20 accordance with the present invention. The server 301 will typically have its 
own operating system 1201. 
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The control software 1202 is for transmitting streaming media data to a 
caching proxy server 401 or to a client 302 using the RTP and RTSP protocols 
and the RTP extensions of the invention. Further, software 1202 receives 
requests from a client 302 or a caching proxy server 401 for streaming media 
5 and negotiates with a caching proxy server 401 for various types of streaming 
media data and associated RTP extensions, and responds to various requests by 
caching proxy servers 401 or clients 302. 

Software 1204 converts original media data 1203, which is usually not in 
a packet format, to a streaming media data format (e.g. packet format) for 

10 transmitting to caching proxy sever 401 or client 302. When converted, the 
converted streaming media data is a representation of the original media data 
1203 that has a different format than the format of the original media data 1203. 

The software 1206 creates RTP extension headers associated with various 
types of streaming media data. The system may assign various ID names and 

15 codes 1205 associated with various RTP extensions to various t5^es of 

streaming media data before its sent to a caching proxy 401 or a client 301. The 
software 1206, in conjimction with software 1202, performs the negotiation 
process with a caching proxy server 401 (or, in some cases where the client asks 
for an RTP extension, such as a security or encryption or authentication 

20 extension, the client) to transmit RTP extension data for an associated streaming 
media data and also performs the transmission process of transmitting 
streaming media data with its associated RTP extension. 
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Figure 13 shows one type of exemplary machine readable media (e.g. 
RAM or hard disk or combination thereof) for storing executable computer 
program instructions for a client server 302 that may be used in accordance 
with the present invention. The client server 302 will t)^ically have its own 
5 operating system 1301 such as a Macintosh OS, or Windows NT, or Unix, or 
other well-known operating systems. The client's media may also include Web 
Browser software 1303 such as Netscape's Navigator or Microsoft's Internet 
Explorer. 

The streaming media data player software 1302 is for receiving and 
10 playing streaming media data transmitted to the client using the RTF protocol. 
The streaming media data player software 1302 may be Quicktime software 
from Apple computer or the Real Player from Real Networks. The streaming 
media data player software 1302 is typically able to send requests to a caching 
proxy server 401 or a server 301 for various different types of streaming media 
15 data and to receive and present (e.g. display images and produce sound) a 
representation of streaming media data. 

In yet another embodiment the streaming media data player software 
1302 may be able to commimicate and negotiate with a caching proxy server 
401 in order to regulate incoming data traffic to handle its load better (e.g. the 
20 software 1302 may ask a CP server 401 to perform frame thinning). 

In the foregoing specification, the invention has been described with 
reference to specific exemplary embodiments thereof. It will, however, be 
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evident that various modifications and changes may be made thereto without 
departing from broader spirit and scope of the invention as set forth in the 
appended claims. The specification and drawings are, accordingly, to be 
regarded in an illustrative rather a restrictive sense. 
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CLAIMS 

What is claimed is: 



11. A method of producing a representation of a streaming media data at a 

2 caching proxy server, said method comprising: 

3 transmitting a request for streaming media data to be delivered to said 

4 caching proxy server; 

5 transmitting a request for data associated with said streaming media 

6 data, said request including an identifier which represents one of several 

7 possible types of data associated with said streaming media data; 

8 receiving said streaming media data and storing said streaming media 

9 data on a storage device which is capable of being controlled by said 

10 caching proxy server; and 

1 1 receiving said data associated with said streaming media data. 

12. A method as in claim 1 further comprising: 

2 storing said data associated with said streaming media data in said 

3 storage device. 

13. A method for data transmission from a server data processing system , 
2 said method comprising: 
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3 receiving a request for streaming media data, said request including a 

4 request for data associated with said streaming media data, said request 

5 including an identifier which represents one of several possible types of data 

6 associated with said streaming media data; 

7 responding to the request with a response indicating a capability of the 

8 server to support the request; and 

9 sending the requested data associated with said streaming media data. 

14. A method of claim 3, wherein said sending uses a real-time transport 
2 protocol (RTF). 

1 5. A method of claim 3, wherein said request may be made by a caching 

2 proxy server or a client. 

16. A method of claim 3, wherein the server responding with an echo only if 
2 it supports the request. 

17. A method of claim 3, further comprising sending the requested data 

2 associated with the transmission protocol in an extensible extended header 

3 format. 
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18. A method of claim 7, wherein the extensible extended header comprises 

2 an extension name and an extension identification (ID) associated with each 

3 separate RTF extension. 

19. A method of claim 3, wherein request may be for one or more t3^e of 
2 transmission protocol data at a time. 

1 10. A method of claim 3, wherein the response by the server comprising 

2 response for each supported transmission protocol data and no response for 

3 any unsupported transmission protocol data. 

1 11. A method of claim 3, further comprising receiving a request to send the 

2 transmission protocol data after sending a response for supported data, and 

3 sending orUy the requested and supported transmission protocol data. 

1 12. A method for operating a caching proxy server comprising: 

2 sending a request for streaming media data to a server, said request 

3 including a request for data associated with said streaming media data, said 

4 request including an identifier which represents one of several possible types of 

5 data associated with said streaming media data; 

6 receiving a response from the server indicating support for the requested 

7 streaming media data; 



47 



8 



informing the server to send the supported data associated with said 



9 streaming media data; 
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receiving the streaming media data from the server; 
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receiving a request from the client to send streaming media data; and 
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sending the requested streaming media data to the client. 



1 13. A method of claim 12, wherein said receiving and sending uses a real- 

2 time transport protocol (RTP). 

1 14. A method of claim 12, wherein said receiving streaming media data from 

2 the server is in an extensible extended header format. 

1 15. A method of claim 12, wherein said sending a request may be for one or 

2 more various and unrelated types of streaming media data to be sent at a time. 

1 16. A method of claim 12, wherein said response from the server comprising 

2 response for each supported type of streaming media and no response for any 

3 unsupported types of streaming media data. 

1 17. A method of claim 14, wherein said extensible extended header format 

2 is appended before sending to client. 
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1 18. A method of claim 17, wherein, appending comprising stripping of name 

2 and ID part of the extensible extended header. 

1 19. A method of claim 12, further comprising determining if a requested 

2 type of streaming media data, that which is required by a caching proxy server 

3 to be able to perform its processes, is missing in the response by the server. 

1 20. A method of claim 19 further comprising terminating the data 

2 transmission process if the requested type of streaming media data is missing in 

3 server's response and is critical to the data transmission process. 

4 21. A method for extending an RTP header comprising: 

5 adding a first RTP sub-extension ID to an RTP header; 

6 defining a length of said first RTP sub-extension by providing a sub- 

7 extension length; 

8 providing data corresponding to the RTP sub-extension ID within said 

9 length defined for said first RTP sub-extension; and 

10 having subsequent RTP sub-extensions following the first RTP sub- 

1 1 extension. 

1 22. A method of claim 21, wherein the length of the RTP sub-extension being 

2 defined by a whole number of 32 bit words. 
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1 23. A method of claim 21, wherein the first RTP sub-extension immediately 

2 following the RTP header. 

1 24. A method of claim 21, wherein RTP sub-extension length is immediately 

2 followed by RTP sub-extension data and immediately preceded by RTP sub- 

3 extension ID. 

1 25. A method of claim 21, wherein the RTP sub-extension contains transmit 

2 time information of each RTP packet. 

1 26. A method of claim 21, wherein the RTP sub-extension contains persistent 

2 ID information. 

1 27. A method of claim 21, wherein the RTP sub-extension contains Frame 

2 Type information. 

1 28. A method of claim 27, wherein the frame type being a 16-bit unsigned 

2 integer value representing a different frame for each value. 

1 29. A method of claim 28, wherein imsigned integer and frame type 

2 comprising: 
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3 assigning an integer value of "0" to an unknown frame type; an integer 

4 value of ''1" to a key frame type; an integer value of "H* to a p-frame type; and 

5 an integer value of "3'" to a b-frame type. 

1 30. A method of claim 29, wherein said key-frame being most important in 

2 priority than any other frames. 

1 31. A method of claim 29, wherein said p-frame being less important in 

2 priority than key-frame, and more important in priority than b-frame. 

1 32. A method of claim 29, wherein said b-frame being less important in 

2 priority than p-frame. 

1 33. A method of claim 29, wherein said b-frame being less important in 

2 priority than key-frame. 

1 34. A method of claim 29, wherein said tmknown-frame being either more 

2 or less important in priority than key-frame, p-frame and b-frame. 

1 35. A method of claim 29, wherein said key-frame being more important in 

2 priority than p-frames, b-frames, and any other frames. 
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1 36. A method of negotiating for various types of streaming media data by 

2 the server comprising: 

3 receiving a request for one or more types of streaming media data from a 

4 caching proxy server or a client, said request including a request for data 

5 associated with said streanung media data, said request including an identifier 

6 which represents one of several possible types of data associated with said 

7 streaming media data; 

8 determining if requested types of streaming media data are supported 

9 by the server; and 

10 responding to the request with a response indicating the capability of the 

1 1 server to support the request. 

1 37. A method of claim 36, further comprising receiving a request to send 

2 supported RTF extensions to the caching proxy or the client. 

1 38. A method claim 37, further comprising responding to request to send 

2 and sending all the supported and requested extensions. 

1 39. A method of claim 36, further comprising receiving a command 

2 terminating the negotiation process. 
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1 40. A method of negotiating for various types of streaming media data by 

2 the caching proxy server comprising: 

3 sending a request for one or more types of related or unrelated 

4 streaming media data to a server, said request including a request for data 

5 associated with said streaming media data, said request including an identifier 

6 which represents one of several possible types of data associated with said 

7 streaming media data; 



8 receiving a response to each requested type of streaming media data; 

9 and 

10 deciding whether to proceed or terminate negotiation process associated 

1 1 with streaming media data. 

1 41. A method of claim 40, wherein deciding comprises: 

2 determining if any requested type of streaming media data is not 

3 supported by server; 

4 checking if unsupported type of streaming media data is essential to 

5 caching proxy server operations; and 

6 sending an execution command to server. 

1 42. A method of claim 40, wherein determining supported types of 

2 streaming media data being performed by checking if a response in a form of 

3 an echo or otherwise has been sent for requested type of streaming media data. 
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1 43. A method of claim 41, wherein execution command being sent based on 

2 results of checking if imsupported type of streaming media data is essential to 

3 caching proxy server operations. 



1 44. A method of 40, wherein decision being to terminate negotiation process. 

1 45. A method of 40, wherein decision being to proceed with negotiation 

2 process and requesting the server to send remaining supported type of 

3 streaming media data. 



1 46. A method of frame thinning by caching proxy server comprising: 

2 receiving a message from a client, said message indicating a need to thin 

3 streaming media data being sent to said client; 

4 evaluating priority of streaming media data; and 

5 sending only selected streaming media data. 

1 47. A method of claim 46, wherein said evaluating comprising: 

2 naming unsigned integers to frame types; 

3 assigning an integer value of ''0" to an imknown frame type; an integer 

4 value of ''V to a key frame type; an integer value of "2'' to a p-frame type; and 

5 an integer value of "3" to a b-frame type. 
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1 48, A method of claim 47, wherein said key-frame being most important in 

2 priority than any other frames. 

1 49. A method of claim 47, wherein said p-frame being less important in 

2 priority than key-frame, and more important in priority than b-frame. 

1 50. A method of claim 47, wherein said b-frame being less important in 

2 priority than p-frame. 

1 51. A method of claim 47, wherein said b-frame being less important in 

2 priority than key-frame. 

1 52. A method of claim 47, wherein said unknown-frame being either more 

2 or less important in priority than key-frame, p-frame and b-frame. 

1 53. A method of claim 47, wherein said key-frame being more important in 

2 priority than p-frames, b-frames, and any other frames. 

1 54. A method of claim 46, further comprising: 

2 receiving a second message from a client to further thin streaming 

3 media data; 
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4 processing message and eliminating more selected streaming media data 

5 and sending streaming media data of higher priority. 

1 55. A method of claim 54, wherein said selected streaming media 

2 frame being eliminated being a b-frame, and sending streaming media data 

3 with higher priority than b-frame to the client. 

1 56. A method of claim 54, wherein said streaming media data being 

2 eliminated being a p-frames and b-frame, and sending frames with higher 

3 priority than both p-frames and b-frames to the client. 

1 57. A method of frame thinning by client comprising: 

2 sending a message to a caching proxy server, said message indicating a 

3 need to thin streaming media data received at said client; 

4 receiving media back from said caching proxy server that are higher in 

5 order than low order streaming media data. 

1 58. A method of claim 57, further comprising: 

2 sending a subsequent message from a client to further thin streaming 

3 media data; 

4 receiving streanung media data that is higher in order then previous 

5 streaming media data received. 
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1 59. A method of claim 57, further comprising: 

2 assigning an iinsigned integer to a frame associated with streaming 

3 media data, wherein said assigning further comprising assigning an integer 

4 value of ''0" to an unknown frame type; an integer value of "1'' to a key frame 

5 type; an integer value of "2" to a p-frame type; and an integer value of ''3" to a 

6 b-frame type. 

1 60. A method of claim 59, wherein said key-frame being most important in 

2 priority than any other frames. 

1 61. A method of claim 59, wherein said p-frame being less important in 

2 priority than key-frame, and more important in priority than b-frame. 

1 62. A method of claim 59, wherein said b-frame being less important in 

2 priority than p-frame. 

1 63. A method of claim 59, wherein said b-frame being less important in 

2 priority than key-frame. 

1 64. A method of claim 59, wherein said unknown-frame being either more 

2 or less important in priority than key-frame, p-frame and b-frame. 
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1 65. A method of claim 59, wherein said key-frame being more important in 

2 priority than p-frames, b-frames, and any other frames. 

1 66. A method of claim 58, wherein said sending comprising eliminating p- 

2 frames and sending selected streaming media data that is higher in order than 

3 p-frames to the client. 

1 67. A method of claim 58, wherein said sending comprising eliminating both 

2 p-frames and b-frames, and sending selected streaming media data that is 

3 higher in order than both p-frames and b-frames. 

1 68. A method of using transmit time of RTP packet transmissions at a 

2 caching proxy server said method comprising: 

3 requesting data corresponding to transmit time for streaming media data 

4 from a server; 

5 receiving said streaming media data and corresponding transmit time 

6 information from the server; 

7 storing the received information; and 

8 transmitting from said caching proxy server to a client said streaming 

9 media data at times specified by said transmit time. 



58 



1 69. A machine-readable medium that provides executable instructions, 

2 which when executed by a set of processors, cause said set of processors to 

3 perform operations for producing a streaming media data at a caching proxy 

4 server comprising: 

5 transmitting a request from for streaming media data to be delivered to 

6 said caching proxy server; 

7 transmitting a request for data associated with said streaming media 

8 data, said request including an identifier which represents one of several 

9 possible types of data associated with said streaming media data; 

10 receiving said streaming media data and storing said streaming media 

1 1 data on a storage device which is capable of being controlled by said 

12 caching proxy server; and 

13 receiving said data associated with said streaming media data. 

1 70. A machine-readable medium as in claim 69 further comprising: 

2 storing said data associated with said streaming media data in said storage 

3 device. 

1 71 . A machine-readable medium that provides executable instructions, 

2 which when executed by a set of processors, cause said set of processors to 

3 perform data transmission operations from a server data processing system 

4 comprising: 
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5 receiving a request for streaming media data, said request including a 

6 request for data associated with said streaming media data, said request 

7 including an identifier which represents one of several possible types of data 

8 associated with said streaming media data; 

9 responding to the request with a response indicating a capability of said 

10 server to support the request; and 

1 1 sending the requested data associated with said streaming media data. 

1 72. A machine-readable medium as in claim 71, wherein said sending uses a 

2 real-time transport protocol (RTF). 

1 73. A machine-readable medium as in claim 71, wherein said request may be 

2 made by a caching proxy server or a client. 

1 74. A machine-readable medium as in claim 71, wherein said responding 

2 with a response occurring only if said server supports the request. 

1 75. A machine-readable medium as in claim 71, further comprising sending 

2 the requested data associated with the transmission protocol in an extensible 

3 extended header format. 
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1 76. A machine-readable mediiim as in claim 75, wherein said extensible 

2 extended header comprises an extension name and an extension identification 

3 (ID) associated with each separate RTF extension. 

1 77. A machine-readable medium as in claim 71, wherein said request may be 

2 for one or more type of transmission protocol data at a time. 

1 78. A machine-readable medium as in claim 71, wherein said response by 

2 the server comprising response for each supported transmission protocol data 

3 and no response for any unsupported transmission protocol data. 

1 79. A machine-readable medium as in claim 71, further comprising receiving 

2 a request to send the transmission protocol data after sending a response for 

3 supported data, and sending only the requested and supported transmission 

4 protocol data. 

1 80. A machine-readable medium that provides executable instructions, 

2 which when executed by a set of processors, cause said set of processors to 

3 perform data transmission receiving operations from a sever comprising: 

4 sending a request for streaming media data to said server, said request 

5 including a request for data associated with said streaming media data, said 
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6 request including an identifier which represents one of several possible types of 

7 data associated with said streaming media data; 

8 receiving a response from said server indicating support for the 

9 requested streaming media data; 

10 informing said server to send the supported data associated with said 

1 1 streaming media data; 

12 receiving the supported streaming media data from said server; 

13 receiving a request from a client to send streaming media data; and 

14 sending the requested streaming media data to said client. 

1 81. A machine-readable medium as in claim 80, wherein said receiving and 

2 sending uses a real-time transport protocol (RTF). 

1 82. A machine-readable medium as in claim 80, wherein said receiving 

2 streaming media data from the server is in an extensible extended header 

3 format. 

1 83. A machine-readable medium as in claim 80, wherein said sending a 

2 request may be for one or more various and unrelated types of streaming media 

3 data to be sent at a time. 
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1 84. A machine-readable medium as in claim 80, wherein said response from 

2 the server comprising response for each supported type of streaming media 

3 and no response for any unsupported types of streaming media data. 

1 85. A machine-readable medium as in claim 82, wherein said extensible 

2 extended header format is appended before sending to client. 

1 86. A machine-readable medium as in claim 85, wherein, appending 

2 comprising stripping of name and ID part of the extensible extended header. 

1 87. A machine-readable medium as in claim 80, further comprising 

2 determining if a requested type of streaming media data, that which is required 

3 by a caching proxy server to be able to perform its processes, is missing in the 

4 response by the server. 

1 88. A machine-readable mediimi as in claim 87, further comprising 

2 terminating the data transmission process if the requested type of streaming 

3 media data is missing in server's response and is critical to the data 

4 transmission process. 
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1 89. A machine-readable medium that provides executable instructions, 

2 which when executed by a set of processors, cause said set of processors to 

3 perform RTP header extending operations comprising: 

4 adding a first RTP sub-extension ID to an RTP header; 

5 defining a length of the first RTP sub-extension by providing a sub- 

6 extension length; 

7 providing data corresponding to the RTP sub-extension ID within said 

8 length defined for said first RTP sub-extension; and 

9 having subsequent RTP sub-extensions following the first RTP sub- 
10 extension. 

1 90. A machine-readable medium as in claim 89, wherein said length of said 

2 RTP sub-extension being defined by a whole number of 32 bit words. 

1 91 . A machine-readable medium as in claim 89, wherein said first RTP sub- 

2 extension immediately following the RTP header. 

1 92. A machine-readable medium as in claim 89, wherein said RTP sub- 

2 extension length is immediately followed by RTP sub-extension data and 

3 immediately preceded by RTP sub-extension ID. 
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1 93. A machine-readable medium as in claim 89, wherein said RTF sub- 

2 extension contains transmit time information of each RTF packet* 

1 94. A machine-readable medium as in claim 89, wherein said RTF sub- 

2 extension contains persistent ID information. 

1 95. A machine-readable medium as in claim 89, wherein said RTF sub- 

2 extension contains RTF Frame Type information. 

1 96. A machine-readable medium as in claim 95, wherein said frame type 

2 being a 16-bit unsigned integer value representing a different frame for each 

3 value. 

1 97. A machine-readable medium as in claim 96, wherein said imsigned 

2 integer and frame type further comprising steps of: 

3 assigning an integer value of ''0'' to an unknown frame type; an integer value 

4 of "1'' to a key frame type; an integer value of "2'' to a p-frame type; and an 

5 integer value of "3'' to a b-frame type. 

1 98. A machine-readable medium as in claim 97, wherein said key-frame 

2 being most important in priority than any other frames. 
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1 99. A machine-readable medium as in claim 97, wherein said p-frame being 

2 less important in priority than key-frame, and more important in priority than 

3 b-frame. 

1 100. A machine-readable medium as in claim 97, wherein said b-frame being 

2 less important in priority than p-frame. 

1 101 . A machine-readable medium as in claim 97, wherein said b-frame being 

2 less important in priority than key-frame. 

1 102. A machine-readable medium as in claim 97, wherein said unknown- 

2 frame being either more or less important in priority than key-frame, p-frame 

3 and b-frame. 

1 103. A machine-readable medium as in claim 97, wherein said key-frame 

2 being more important in priority than p-frames and b-frames other frames. 

1 104. A machine-readable medium that provides executable instructions, 

2 which when executed by a set of processors, cause said set of processors to 

3 perform negotiating operations for various types of streaming media data by a 

4 server comprising: 
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5 receiving a request for one or more types of streaming media data from a 

6 caching proxy server or a client, said request including a request for data 

7 associated with said streaming media data, said request including an identifier 

8 which represents one of several possible types of data associated with said 

9 streaming media data; 

10 determining if requested types of streaming media data are supported 

11 by the server; and 

12 responding to the request with a response indicating the capability of the 

13 server to support the request. 

1 105. A machine-readable medium as in claim 104, further comprising 

2 receiving a request to send supported RTP extensions to the caching proxy or 

3 the client. 

1 106. A machine-readable medium as in claim 105, further comprising 

2 responding to request to send and sending all the supported and requested 

3 extensions. 

1 107. A machine-readable medium as in claim 104, further comprising 

2 receiving a command terminating the negotiation process. 
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1 108. A machine-readable medium that provides executable instructions, 

2 which when executed by a set of processors, cause said set of processors to 

3 perform negotiating operations for various types of streaming media data by a 

4 caching proxy server comprising: 

5 sending a request for one or more types of related or imrelated 

6 streaming media data to a server, said request including a request for data 

7 associated with said streaming media data, said request including an identifier 

8 which represents one of several possible types of data associated with said 

9 streaming media data; 

10 receiving a response to each requested type of streaming media data; 

11 and 

12 deciding whether to proceed or terminate negotiation process associated 

13 with streaming media data. 



1 109. A machine-readable medium as in claim 108, wherein deciding 

2 comprises: 

3 determining if any requested type of streaming media data is not 

4 supported by server; 

5 checking if unsupported type of streaming media data is essential to 

6 caching proxy server operations; and 

7 sending an execution command to server. 
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1 110. A machine-readable medium as in claim 108, wherein determining 

2 supported types of streaming media data being performed by checking if a 

3 response in a form of an echo or otherwise has been sent for requested type of 

4 streaming media data. 

1 111. A machine-readable medium as in claim 109, wherein said execution 

2 command being send based on results of checking if unsupported type of 

3 streaming media data is essential to caching proxy server operations. 

1 112. A machine-readable medium as in claim 108, wherein said decision 

2 being to terminate negotiation process. 

1 113. A machine-readable medium as in claim 108, wherein said decision 

2 being to proceed with negotiation process and requesting the server to send 

3 remaining supported type of streaming media data. 

1 114. A machine-readable medium that provides executable instructions, 

2 which when executed by a set of processors, cause said set of processors to 

3 perform frame thinning operations by a caching proxy server comprising: 

4 receiving a message from a client to thin frames in a streaming media 

5 data transmission from said caching proxy server; 

6 evaluating priority of frames; and 
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7 sending only selected frames. 

1 115. A machine-readable medium as in claim 114, wherein said frame priority 

2 being evaluated by naming imsigned integers to frame types, said unsigned 

3 integers comprising: 

4 assigrung an integer value of 'V to an imknown frame type; an integer value 

5 of 'T' to a key frame type; an integer value of "2'' to a p-frame type; and an 

6 integer value of "3'' to a b-frame type. 

1 116. A machine-readable medium as in claim 114, wherein said key-frame 

2 being most important in priority than any other frames. 

1 117. A machine-readable medium as in claim 114, wherein said p-frame being 

2 less important in priority than key-frame, and more important in priority than 

3 b-frame. 

1 118. A machine-readable medium as in claim 114, wherein said b-frame being 

2 less important in priority than p-frame. 

1 119. A machine-readable medium as in claim 114, wherein said b-frame being 

2 less important in priority than key-frame. 
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1 120. A machine-readable medium as in claim 114, wherein said unknown- 

2 frame being either more or less important in priority than key-frame, p-frame a 

3 b-frame. 

1 121. A machine-readable medium as in claim 114, wherein said key-frame 

2 being more important in priority than p-frames, b-frames, and any other 

3 frames. 

1 122. A machine-readable medium as in claim 114, further comprising: 

2 receiving a second request from a client to further thin frames; 

3 processing request and eliminating more selected frames and sending 

4 frames of higher priority. 

1 123. A machine-readable medium as in claim 122, wherein said selected 

2 frame being eliminated being a p-frame, and sending frames with higher 

3 priority than p-frame to the client. 

1 124. A machine-readable medium as in claim 122, wherein said selected 

2 frame being eliminated being a p-frames and b-frame, and sending frames with 

3 higher priority than both p-frames and b-frames to the client. 
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1 125. A machine-readable medium that provides executable instructions, 

2 which when executed by a set of processors, cause said set of processors to 

3 perform frame thinning operations by a client comprising: 

4 sending a thinning message to a caching proxy server indicating not to 

5 send low order frames; 

6 receiving frames back from caching proxy server that are higher in order 

7 than low order frames. 

1 126. A machine-readable medium as in claim 125, further comprising: 

2 sending a subsequent message from a client to further thin frames; 

3 receiving frames that are higher in order then previous frames received. 

1 127. A machine-readable medium as in claim 125, wherein said frames being 

2 assigned imsigned integers, said imsigned integers comprising: 

3 assigning an integer value of "0'' to an unknown frame type; an integer value 

4 of to a key frame type; an integer value of "2" to a p-frame type; and an 

5 integer value of ''3'' to a b-frame type. 

1 128. A machine-readable medium as in claim 125, wherein said key-frame 

2 being most important in priority than any other frames. 
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1 129. A machine-readable medium as in claim 125, wherein said p-frame being 

2 less important in priority than key-frame, and more important in priority than 

3 b-frame. 

1 130. A machine-readable mediiun as in claim 125, wherein said b-frame being 

2 less important in priority than p-£rame. 

1 131 . A machine-readable medium as in claim 125, wherein said b-frame being 

2 less important in priority than key-frame. 

1 132. A machine-readable medium as in claim 125, wherein said imknown- 

2 frame being either more or less important in priority than key-frame, p-frame 

3 and b-frame. 

1 133. A machine-readable medium as in claim 125, wherein said key-frame 

2 being more important in priority than p-frames, b-frames and any other frames. 

1 134. A machine-readable medium as in claim 126, wherein said eliminating 

2 comprising eliminating p-frames and sending selected frames that are higher in 

3 order than p-frames to the client. 
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1 135. A machine-readable medium as in claim 126, wherein said eliminating 

2 comprising eliminating both p-frames and 

3 b-frames, and sending selected frames that are higher in order than both p- 

4 frames and b-frames. 



1 136. A machine-readable medium that provides executable instructions, 

2 which when executed by a set of processors, cause said set of processors to send 

3 transmit time of RTP packet transmission operations from a caching proxy 

4 comprising: 

5 requesting data corresponding to transmit time for streaming media data 

6 from a server; 

7 receiving said streaming media data corresponding with transmit time 

8 information from the server; 

9 storing the received information; and 

10 transmitting from said caching proxy server to a client, said streaming 

1 1 media data at times specified by said transmit time. 

1 137. A caching proxy server comprising: 

2 means for transmitting a request for streaming media data to be 

3 delivered to said caching proxy server; 
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4 means for transmitting a request for data associated with said streaming 

5 media data, said request including an identifier which represents one of 

6 several possible types of data associated with said streaming media data; 

7 means for receiving said streaming media data and storing said 

8 streaming media data on a storage device which is capable of being 

9 controlled by said caching proxy server; and 

10 means for receiving said data associated with said streaming media data. 



1 138. A server data processing system comprising: 

2 means for receiving a request for streaming media data, said request 

3 including a request for data associated with said streaming media data, said 

4 request including an identifier which represents one of several possible t5^es of 

5 data associated with said streaming media data; 

6 means for responding to the request with a response indicating a 

7 capability of the server to support the request; and 

8 means for sending the requested data associated with said streaming 

9 media data. 



1 139. A caching proxy server comprising: 

2 means for sending a message for streaming media data to a server, said 

3 request including a request for data associated with said streaming media data. 
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4 said request including an identifier which represents one of several possible 

5 types of data associated with said streaming media data; 

6 means for receiving a response from the server indicating support for the 

7 requested streaming media data; 

8 means for informing the server to send the supported data associated 

9 with said streaming media data; 

10 means for receiving the streaming media data from the server; 

1 1 means for receiving a request from the client to send streaming medi'a 

12 data; and 

13 means for sending the requested streaming media data to the client. 

1 140. A RTF header comprising: 

2 means for adding a first RTF sub-extension ID to an RTF header; 

3 means for defining a length of said first RTF sub-extension by providing 

4 a sub-extension length; 

5 means for providing data corresponding to the RTF sub-extension ID 

6 within said length defined for said first RTF sub-extension; and 

7 means for having subsequent RTF sub-extensions following the first RTF 

8 sub-extension. 

1 141. A server comprising: 
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2 means for receiving a request for one or more types of streaming media 

3 data from a caching proxy server or a client, said request including a request for 

4 data associated with said streaming media data, said request including an 

5 identifier which represents one of several possible types of data associated with 

6 said streaming media data; 

7 means for determining if requested types of streaming media data are 

8 supported by the server; and 

9 means for responding to the request with a response indicating the 



10 capability of the server to support the request. 



1 142. A caching proxy server comprising: 

2 means for sending a request for one or more types of related or 

3 unrelated streaming media data to a server, said request including a request for 

4 data associated with said streaming media data, said request including an 

5 identifier which represents one of several possible types of data associated with 

6 said streaming media data; 



7 means for receiving a response to each requested type of streaming 

8 media data; and 

9 means for deciding whether to proceed or terminate negotiation process 
10 associated with streaming media data. 

1 143. A frame thinning caching proxy server comprising: 
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2 means for receiving a message from a client to thin frames in a streaming 

3 media data transmission from said caching proxy server; 

4 means for evaluating priority of frames; and 

5 means for sending only selected frames. 



1 144. A client comprising: 

2 means for sending a message to a caching proxy server, said message 

3 indicating a need to thin streanung media data received at said client; 

4 means for receiving media back from caching proxy server that are 

5 higher in order than low order media. 



1 145. A caching proxy server comprising: 

2 means for requesting data corresponding to transmit time for streaming 

3 media data from a server; 

4 means for receiving said streaming media data and corresponding 

5 transmit time information from the server; 

6 means for storing the received information; and 

7 means for transmitting from said caching proxy server to a client said 

8 streaming media data at times specified by said transmit time. 
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ABSTRACT 

The present invention provides several methods and apparatuses for 
transmitting multimedia data using streaming media protocols such as real- 
time transfer protocols (RTF) and real-time streaming protocols (RTSP) in a 
computer network environment. In one exemplary embodiment, a request for 
RTF data and its associated extension is sent from the caching proxy server to 
the server. The request may be for one specific type of data or multiple 
unrelated types of data. The server responds to the request indicating its 
support for the requested RTF extension data. The caching proxy server 
determines whether to proceed or terminate the data transmission process 
based on the response provided by the server. If it is determined to proceed 
with the data transmission process, the caching proxy informs the server to 
send the requested and supported RTF data. The server sends the requested 
data in a variable and extendible header format. 
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Figure 1-A 



Originating server and caching proxy server (CP) 
negotiate, through real-time streaming protocol 

(RTSP) for server to send and CP server to receive 
streaming media data (For example: QuickTime 
Streaming Audio Data). 



CP server receives streaming media data, using real 
time transfer protocol (RTP), in streaming media 
format not the same format as stored on originating 
server and CP server stores the streaming media data, 
in the streaming media format, on a storage device 
controlled by CP server. 



CP server receives request, through RTSP, from a 
client for the streaming media data and retrieves the 
data from the storage device and transmits the 
streaming media data to the client system. 
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CP asks SETUP in RTSP 
For Audio data and specifies 
extensions to RTP data 
for audio data 



Server Responds / Negotiates 



CP asks SETUP in RTSP 
For Video data and specifies 
extensions to RTP data 
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Server Responds / Negotiates 



CP asks PLAY in RTSP 



Server responds with RTP data 
for all tracks with negotiated 
extensions 
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FIGURE - 6 





FIGURE -7 



CP asks for transmit time 
information from the server 



Server receives the request for 
Transmit time fTT) 



Server responds and sends TT 
identifier code, which 
corresponds, to TT type of 
extension, in an extensible 
header format - Streaming data 
is sent with TT extended data 



- Extensible header format is a 
sub-extension containing a sub- 
extension ID, identifying the 
content of the information. 

- TT sub-extension also 
contains a single 64-bit 
unsigned integer representing 
recommended transmission 
time of RTP packet in 
milliseconds 



no- 



Server does not respond to 
request. This indicates that 
Server does not support the 
extension requested 
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CP may terminate process 
If TT is critical element 
of the CP operation 
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CP gets the TT data, removes 
extension header and stores the 
data locally; TT times are 
associated with appropriate 
streaming data 



Client requests streaming data 
by making a PLAY request in 
RTSP 



no 
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Client may also 
ask for PLAY or 
other data directly 
from server 



CP responds and sends tracks of data at appropriate times 
specified by TT times associated with streaming data 
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Client PLAYS data 



CP asks for frame type 
information from the server 



Server receives the request for 
Frame tvpe (FT) 



Server responds and sends FT 
identifier code, which 
corresponds, to FT type of 
extension, in an extensible 
header format - Streaming data 
is sent with FT extended data 



- Extensible header format is a 
sub-extension containing a sub- 
extension ID, identifying the 
content of the information. FT 
sub-extension also contains 16- 
bit unsigned integer value with 
well-known frame types such as 
0-unknown frame, 1 -key frame, 
2-b frame, and 3-p frame. Future 
versions can add more frames. 



CP 



CP gets the FT data, removes 
extension header and stores the 
data locally; FT frames are 
associated with appropriate 
streaming data 



CP reviews client request Stops 
sending least important frames 
and sends only higher 
importance frames 
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Server does not respond to 
request. This indicates that 
Server does not support the 
extension requested 



CP may terminate process 
If FT is critical element 
of the CP operation 
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Client informs CP that it is 
overloaded with information 
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Client may also 
ask for FT or 
other data directly 
from the server 
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CP asks by name for one or more RTP 
sub-extensions using RTSP protocol 



^0\ 



Server responds back to CP indicating its 
support for the requested sub-extensions. 

Server also transmits to CP an identifier (e.g 
number code) corresponding to each name 

of RTP extension. CP may later use the code 
when receiving extended data. Response 
may or may not be a complete response. 
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CP receives server's response 



CP determines whether server responded 
to all requested sub-extensions 
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Server responded to all 
requested sub-extensions 
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Server did not respond to all 
requested sub-extensions 
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associated the RTP sub-extensions 



Server sends all supported RTP sub- 
extensions and streaming media data 
associated with the RTP sub- 
extensions 



Are any of the missing sub- 
extensions critical to CP 
processes? 



CP stores sub-extension data and 
streaming media associated with the 
RTP sub-extension locally 



CP may remove sub-extension fields 
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CP decides whether 
to terminate 
operations 



CP evaluates client requests and 
performs functions accordingly. 
For example frame thinning or using 
transmit time optionally to transmit 
data 



CP sends streaming media data to client 
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APPENDIX B 



Title 37, Code of Federal Regulations, Section 1 .56 
Duty to Disclose Information Material to Patentability 

(a) A patent by its very nature is affected with a public interest. The public interest is best served, 
and the most effective patent examination occurs when, at the time an application is being examined, the 
Office Is aware of and evaluates the teachings of all Information material to patentability. Each individual 
associated with the filing and prosecution of a patent application has a duty of candor and good faith in 
dealing with the Office, which includes a duty to disclose to the Office all information known to that individual 
to be material to patentability as defined in this section. The duty to disclosure information exists with respect 
to each pending claim until the claim is cancelled or withdrawn from consideration, or the application becomes 
abandoned. Information material to the patentability of a claim that is cancelled or withdrawn from 
consideration need not be submitted if the information is not material to the patentability of any claim 
remaining under consideration in the application. There is no duty to submit information which is not material 
to the patentability of any existing claim. The duty to disclosure all information known to be material to 
patentability is deemed to be satisfied if all information known to be material to patentability of any claim 
issued in a patent was cited by the Office or submitted to the Office in the manner prescribed by §§1 .97(b)-(d) 
and 1 .98. However, no patent will be granted on an application in connection with which fraud on the Office 
was practiced or attempted or the duty of disclosure was violated through bad faith or intentional misconduct. 
The Office encourages applicants to carefully examine: 

(1) Prior art cited in search reports of a foreign patent office in a counterpart application, and 

(2) The closest information over which individuals associated with the filing or prosecution of a 
patent application believe any pending claim patentably defines, to make sure that any material information 
contained therein is disclosed to the Office. 

(b) Under this section, information is material to patentability when it is not cumulative to 
information already of record or being made or record in the application, and 

(1 ) It establishes, by itself or in combination with other infomnation, a prima facie case of 
unpatentability of a claim; or 

(2) It refutes, or is inconsistent with, a position the applicant takes in: 

(i) Opposing an argument of unpatentability relied on by the Office, or 

(ii) Asserting an argument of patentability. 

A prima facie case of unpatentability is established when the information compels a conclusion that a claim is 
unpatentable under the preponderance of evidence, burden-of-proof standard, giving each term in the claim 
its broadest reasonable construction consistent with the specification, and before any consideration is given to 
evidence which may be submitted in an attempt to establish a contrary conclusion of patentability. 

(c) individuals associated with the filing or prosecution of a patent application within the 
meaning of this section are: 

(1 ) Each inventor named in the application; 

(2) Each attorney or agent who prepares or prosecutes the application; and 

(3) Every other person who is substantively involved in the preparation or prosecution of the 
application and who is associated with the inventor, with the assignee or with anyone to whom there is an 
obligation to assign the application. 

(d) Individuals other than the attorney, agent or inventor may comply with this section by 
disclosing information to the attorney, agent, or inventor. 
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